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DETAILED ACTION 



Claims 1-28 are pending in this action. 



Claim Rejections - 35 USC ' 103 



2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



3. Claims 1-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Seller (U.S. 
Patent 6,046,712) in view of Ohshima (U.S. 6,522,312), and further in view of Ralston (U.S. 



Regarding independent claim 1, Beller teaches a method of viewing data and directing 
the view of a wearable camera within the field of view of the operator by teaching a head 
mounted display system that allows an operator of a remote system to see what the user of the 
head mounted system is viewing wherein the head mounted communication system includes a 
support to be worn on a user's head and a camera system that is mounted on the support to pick 
up an image within at least a portion of the user's field of view {see column 1, lines 12-17, 48- 



Furthermore, Beller teaches how to select data from memory storage by teaching how the 
head mounted communication system 10 operates to transmit video and/or audio data to a 
remote assistant's terminal 13 in accordance with the flow chart depicted in FIG. 4 wherein at 
block 140, the microprocessor 90 determines whether there is a frame of compressed video data 



6,094,625). 



57, 62-66). 
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in the buffer memory 86 to be transmitted, and if so, the microprocessor 90 proceeds to block 
142 to retrieve the compressed data for a video frame (column 9, lines 21-28, figure 4 at 86, 
140, 142). 

Also, Beller teaches how to display data on the wearable display worn by the operator 
(column 4, lines 19-22, figure 3 at 14). 

Furthermore, Beller teaches how to detect one or more visual markers by teaching how 
the user can realign his view to the view of the picked up image marked by the assistant by 
merely moving his head until the user's displayed image coincides with the real world image 
seen through the optics of the system (see column 2, lines 39-59; see also column 3, lines 64 
through column 4, lines 5). 

However, Beller does not teach how to automatically detect one or more visual markers 
within the image through the use of pattern recognition. On the other hand, Ohshima teaches 
this concept by teaching markers with circular and square patterns wherein the markers sensed 
by the CCD camera 240 attached to each player detects the position in the image (column 13, 
lines 41-53, figure 5). 

Thus, it would have been obvious to a person of ordinary skill in the art to combine 
Beller and Ohshima because while Beller teaches a method of viewing data and directing the 
view of a wearable camera within the field of view of the operator, Ohshima teaches how to 
automatically detect one or more visual markers within the image through the use of pattern 
recognition by teaching markers with circular and square patterns wherein the markers sensed 
by the CCD camera 240 attached to each player detects the position in the image (column 13, 
lines 41-53, figure 5). The motivation for combining these inventions would have been to 
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achieve an improvement of precise detection of the head position and/or posture of an operator 
to which mixed reality is presented (column 1, lines 9-12). 

However, Beller and Ohshima do not how to select data from a database memory storage 
having a predefined association with the markers. On the other hand, Ralston teaches this 
concept by teaching how positions and attributes of real objects are stored in a database memory 
carried by the operator (see Abstract), and these are associated with points that are indicated by 
virtual markers in order to facilitate the operation of a user (column 13, lines 16-37, figure 13). 

Thus, it would have been obvious to a person of ordinary skill in the art to combine 
Beller and Ralston because while Beller teaches a method of viewing data and directing the 
view of a wearable camera within the field of view of the operator by teaching a head mounted 
display system that allows an operator of a remote system to see what the user of the head 
mounted system is viewing wherein the head mounted communication system includes a 
support to be worn on a user's head and a camera system that is mounted on the support to pick 
up an image within at least a portion of the user's field of view (see column 1, lines 12-17, 48- 
57, 62-66), Ralston teaches how a memory storage is in communication with the network 
containing the information associated with markers by teaching how positions and attributes of 
real objects are stored in a database memory carried by the operator (see Abstract), and these 
are associated with points that are indicated by virtual markers in order to facilitate the 
operation of the worker (column 13, lines 16-37, figure 13). The motivation for combining 
these inventions would have been to facilitate determining a current field of view of the 
operator through the display using the position and head orientation measurements, accessing a 
database containing information on a predetermined layout of a plurality of points, lines, 
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surfaces or other features in the design, generating an image containing representations of one 
or more of the features in the layout which are within the current field of view, presenting the 
image to the operator on the display with the representations superimposed on their 
predetermined positions in the field of view, and generating and presenting subsequent images 
on the display as the operator moves around the site and observes the design from different 
viewpoints (column 4, lines 13-24). 

Regarding claims 2 and 3, in further discussion of claim 1, Beller teaches how the head 
mounted communication system includes a support to be worn on a user's head and a camera 
system that is mounted on the support to pick up an image within at least a portion of the user's 
field of view {see column 1, lines 62-66). 

Regarding claim 4, in further discussion of claim 1, Beller teaches how the invention 
allows the user of the head mounted system to view video inputs received from the remote 
system so that they appear to be superimposed upon the real world {see Abstract; column 1, 
lines 12-17). 

Regarding claim 5, in further discussion of claim 1, Beller teaches an input device that is 
actuable by the user to provide a user input command for selecting, for display on the head 
mounted display screen, data received from the remote system and the non-remote data sources 
(column 2, lines 60-67). 
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Regarding claim 6, in further discussion of claim 1, Beller teaches how the operator of 
the remote system can see the changing view of the user of the head mounted system in real 
time such that the operator can add marks or other inputs to the real time image depicted on his 
display so as to identify , for example, a particular object within the picked up image of the real 
world wherein these marks can then be transmitted to the head mounted system in real time 
(column 2, lines 16-38). 

Regarding claim 7, in further discussion of claim 5, Beller teaches how his invention 
relates to a head mounted display system and more particularly to a head mounted 
communication system for providing interactive visual and/or audio communications between a 
user of the head mounted system and an operator of a remote system (column 1, lines 8-12). 

Regarding claim 8, in further discussion of claim 5, Beller teaches the display 19 
includes a touch responsive panel so that a stylus 23 can be used therewith to allow the assistant 
to mark the display by drawing circles, arrows, etc., that are useable to identify objects within 
the displayed image such that the touch panel and associated processor within the terminal 13 
are responsive to contact by the stylus 23 so as to control the display 19 to cause a visual 
representation of the marks to be generated on the display as if they were drawn by the stylus 23 
(column 3, lines 64 through column 4, lines 5). 



Regarding claim 9, in further discussion of claim 5, Beller teaches a method of viewing 
data and directing the view of a wearable camera within the field of view of the operator by 
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teaching a head mounted display system that allows an operator of a remote system to see what 
the user of the head mounted system is viewing wherein the head mounted communication 
system includes a support to be worn on a user's head and a camera system that is mounted on 
the support to pick up an image within at least a portion of the user's field of view {see column 
1, lines 12-17, 48-57, 62-66). 



Regarding independent claim 10, Beller teaches a method of viewing data and directing 
the view of a wearable camera within the field of view of the operator by teaching a head 
mounted display system that allows an operator of a remote system to see what the user of the 
head mounted system is viewing wherein the head mounted communication system includes a 
support to be worn on a user's head and a camera system that is mounted on the support to pick 
up an image within at least a portion of the user's field of view {see column 1, lines 12-17, 48- 
57, 62-66). 

Also, Beller teaches how the operator of the remote system can see the changing view of 
the user of the head mounted system in real time such that the operator can add marks or other 
inputs to the real time image depicted on his display so as to identify , for example, a particular 
object within the picked up image of the real world wherein these marks can then be transmitted 
to the head mounted system in real time (column 2, lines 16-38). 

Also, Beller teaches to determine the location of the operator by means of data 
representing the marks input by the operator, which can be transmitted in association with a 
location determined with respect to the picked up image (column 2, lines 16-38). 
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Furthermore, Beller teaches how to detect one or more visual markers by teaching how 
the user can realign his view to the view of the picked up image marked by the assistant by 
merely moving his head until the user's displayed image coincides with the real world image 
seen through the optics of the system (see column 2, lines 39-59; see also column 3, lines 64 
through column 4, lines 5). 

However, Beller does not teach how to automatically detect one or more visual markers 
within the image through the use of pattern recognition. On the other hand, Ohshima teaches 
this concept by teaching markers with circular and square patterns wherein the markers sensed 
by the CCD camera 240 attached to each player detects the position in the image (column 13, 
lines 41-53, figure 5). 

Thus, it would have been obvious to a person of ordinary skill in the art to combine 
Beller and Ohshima because while Beller teaches a method of viewing data and directing the 
view of a wearable camera within the field of view of the operator, Ohshima teaches how to 
automatically detect one or more visual markers within the image through the use of pattern 
recognition by teaching markers with circular and square patterns wherein the markers sensed 
by the CCD camera 240 attached to each player detects the position in the image (column 13, 
lines 41-53, figure 5). The motivation for combining these inventions would have been to 
achieve an improvement of precise detection of the head position and/or posture of an operator 
to which mixed reality is presented (column 1, lines 9-12). 

However, Beller and Ohshima do not how to select data from a database memory storage 
having a predefined association with the markers. On the other hand, Ralston teaches this 
concept by teaching how positions and attributes of real objects are stored in a database memory 
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carried by the operator (see Abstract), and these are associated with points that are indicated by 
virtual markers in order to facilitate the operation of a user (column 13, lines 16-37, figure 13). 

Thus, it would have been obvious to a person of ordinary skill in the art to combine 
Beller and Ralston because while Beller teaches a method of viewing data and directing the 
view of a wearable camera within the field of view of the operator by teaching a head mounted 
display system that allows an operator of a remote system to see what the user of the head 
mounted system is viewing wherein the head mounted communication system includes a 
support to be worn on a user's head and a camera system that is mounted on the support to pick 
up an image within at least a portion of the user's field of view (see column 1, lines 12-17, 48- 
57, 62-66), Ralston teaches how a memory storage is in communication with the network 
containing the information associated with markers by teaching how positions and attributes of 
real objects are stored in a database memory carried by the operator (see Abstract), and these 
are associated with points that are indicated by virtual markers in order to facilitate the 
operation of the worker (column 13, lines 16-37, figure 13). The motivation for combining 
these inventions would have been to facilitate determining a current field of view of the 
operator through the display using the position and head orientation measurements, accessing a 
database containing information on a predetermined layout of a plurality of points, lines, 
surfaces or other features in the design, generating an image containing representations of one 
or more of the features in the layout which are within the current field of view, presenting the 
image to the operator on the display with the representations superimposed on their 
predetermined positions in the field of view, and generating and presenting subsequent images 
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on the display as the operator moves around the site and observes the design from different 
viewpoints (column 4, lines 13-24). 

Regarding claims 11 and 12, in further discussion of claim 10, Beller teaches how to 
monitor the change of location of the markers within the field of view of the head mounted 
display (column 2, lines 39-59). 

Regarding claim 13, in further discussion of claims 10, Beller teaches how the invention 
allows the user of the head mounted system to view video inputs received from the remote 
system so that they appear to be superimposed upon the real world (see Abstract; column 1, 
lines 12-17). 

Regarding claims 21, 22, 26 and 27, in further discussion of claim 1, Ohshima teaches 
markers with circular and square patterns wherein the markers sensed by the CCD camera 240 
attached to each player detects the position in the image (column 13, lines 41-53, figure 5). 

Regarding claims 23 and 24, in further discussion of claims 1 and 9, Ralston teaches this 
concept by teaching how positions and attributes of real objects are stored in a database memory 
carried by the operator (see Abstract), and these are associated with points that are indicated by 
virtual markers in order to facilitate the operation of a user (column 13, lines 16-37, figure 13). 
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Regarding claims 25, in further discussion of claims 1, Ralston teaches how the view for 
a predetermined period of time is determined according the employee preference and profile 
(see Abstract). 

4. Claims 14-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Beller 
(U.S. Patent 6,046,712) in view of Ralston (U.S. 6,094,625). 

Regarding independent claim 14, and for claims 19 and 20, Beller teaches a method of 
viewing data and directing the view of a wearable camera within the field of view of the 
operator by teaching a head mounted display system that allows an operator of a remote system 
to see what the user of the head mounted system is viewing wherein the head mounted 
communication system includes a support to be worn on a user's head and a camera system that 
is mounted on the support to pick up an image within at least a portion of the user's field of 
view {see column 1, lines 12-17, 48-57, 62-66). It is inherent that this head mounted display 
comprises circuitry and logic that enables it accomplish tasks such as picking up an image 
within at least a portion of the user's field of view. 

Furthermore, Beller teaches how to detect one or more visual markers by teaching how 
the user can realign his view to the view of the picked up image marked by the assistant by 
merely moving his head until the user's displayed image coincides with the real world image 
seen through the optics of the system {see column 2, lines 39-59; see also column 3, lines 64 
through column 4, lines 5). 

Also, Beller teaches how the operator of the remote system can see the changing view of 
the user of the head mounted system in real time such that the operator can add marks or other 
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inputs to the real time image depicted on his display so as to identify , for example, a particular 
object within the picked up image of the real world wherein these marks can then be transmitted 
to the head mounted system in real time (column 2, lines 16-38). 

Also, Beller teaches how to display data on the wearable display worn by the operator 
(column 4, lines 19-22, figure 3 at 14). 

However, Beller does not teach memory storage in communication with the network 
containing the information associated with the markers. On the other hand, Ralston teaches this 
concept by teaching how positions and attributes of real objects are stored in a database memory 
carried by the operator (see Abstract), and these are associated with points that are indicated by 
virtual markers in order to facilitate the operation of the worker (column 13, lines 16-37, figure 
13). 

Thus, it would have been obvious to a person of ordinary skill in the art to combine 
Beller and Ralston because while Beller teaches a method of viewing data and directing the 
view of a wearable camera within the field of view of the operator by teaching a head mounted 
display system that allows an operator of a remote system to see what the user of the head 
mounted system is viewing wherein the head mounted communication system includes a 
support to be worn on a user's head and a camera system that is mounted on the support to pick 
up an image within at least a portion of the user's field of view {see column 1, lines 12-17, 48- 
57, 62-66), Ralston teaches how a memory storage is in communication with the network 
containing the information associated with markers by teaching how positions and attributes of 
real objects are stored in a database memory carried by the operator (see Abstract), and these 
are associated with points that are indicated by virtual markers in order to facilitate the 



Application/Control Number: 10/046,468 Page 13 

Art Unit: 2675 

operation of the worker (column 13, lines 16-37, figure 13). The motivation for combining 
these inventions would have been to facilitate determining a current field of view of the 
operator through the display using the position and head orientation measurements, accessing a 
database containing information on a predetermined layout of a plurality of points, lines, 
surfaces or other features in the design, generating an image containing representations of one 
or more of the features in the layout which are within the current field of view, presenting the 
image to the operator on the display with the representations superimposed on their 
predetermined positions in the field of view, and generating and presenting subsequent images 
on the display as the operator moves around the site and observes the design from different 
viewpoints (column 4, lines 13-24). 

Regarding claim 15, in further discussion of claim 14, Beller teaches how the invention 
allows the user of the head mounted system to view video inputs received from the remote 
system so that they appear to be superimposed upon the real world {see Abstract; column 1, 
lines 12-17). 

Furthermore, Beller teaches how the user would simultaneously view information on the 
display and objects within the field of view of the operator (column 4, lines 28-38). 

Regarding claim 16, in further discussion of claims 10 and 15, Beller teaches how the 
invention allows the user of the head mounted system to view video inputs received from the 
remote system so that they appear to be superimposed upon the real world {see Abstract; 
column 1, lines 12-17). 
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Regarding claims 17 and 18, in further discussion of claim 14, Ralston teaches this 
concept by teaching how positions and attributes of real objects are stored in a database memory 
carried by the operator (see Abstract), and these are associated with points that are indicated by 
virtual markers in order to facilitate the operation of the worker (column 13, lines 16-37, figure 



Regarding claim 28, in further discussion of claim 14, Ralston teaches this concept by 
teaching how positions and attributes of real objects are stored in a database memory carried by 
the operator (see Abstract), and these are associated with points that are indicated by virtual 
markers in order to facilitate the operation of the worker (column 13, lines 16-37, figure 13). 

Response to Arguments 
5. Applicant's amendments and arguments filed July 9, 2004 have been fully considered but 
they are moot in view of the new grounds for rejection. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Uchendu O. Anyaso whose telephone number is (703) 306-5934. 
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Steve 
Saras, can be reached at (703) 305-9720. 

Any response to this action should be mailed to: 



13). 



Contact Information 
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Commissioner of Patents and Trademarks 
Washington, D.C. 20231 
or faxed tof 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, Arlington, 
VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or proceeding should 
be directed to the Technology Center 2600 Customer Service Office whose telephone number is 
(703) 306-0377. 



Uchendu O. Anyaso 
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